Efficient retrieval of anomalous events with priority learning

ABSTRACT

Local models learned from anomaly detection are used to rank detected anomalies. The local models include image feature values extracted from an image field of video image data with respect to different predefined spatial and temporal local units, wherein anomaly results are determined by failures to fit to applied anomaly detection module local models. Image features values extracted from the image field local units associated with anomaly results are normalized, and image feature values extracted from the image field local units are clustered. Weights for anomaly results are learned as a function of the relations of the normalized extracted image feature values to the clustered image feature values. The normalized values are multiplied by the learned weights to generate ranking values to rank the anomalies.

BACKGROUND

The present invention relates to the efficient ranking and selectiveretrieval of anomalous events (anomalies) determined in visual imagedata.

Determining and recognizing anomalous motion activities (anomalies) invisual image data is useful in determining occurrences or absences ofcertain activities or events. For example, image data of structures maybe monitored for changes in expected or normal visual data patterns thatare indicative of events and behaviors diverging from norms, such asimmediate or potential failures of structural components or humanmovements or activities outside of compliance with usual safety or otheractivity processes and policies. If readily distinctive to humananalysis, such anomalies may be identified by capturing and recordingvisual data through still image and video systems for subsequent orcontemporaneous analysis. However, with large amounts of data,discerning anomalies of importance from other anomalies may bedifficult, time consuming or inefficient, and even non-feasible. Moreparticularly, it is not enough to merely recognize that an anomaly hasoccurred in the context of high frequencies or numbers anomalyoccurrences, especially if some otherwise equivalent anomalies may havemore importance than others.

Automated video systems and methods are known wherein computers or otherprogrammable devices directly analyze video data and attempt torecognize anomaly objects, people, events or activities of concern,etc., through identifying anomalous motion patterns through computervision applications. However, discernment of more significant anomaliesfrom other anomalies or even from normal patterns, events, etc., byautomated video surveillance systems and methods systems is often notreliable in realistic, real-world environments and applications due to avariety of factors. For example, visual image data may be difficult toanalyze or vary over time due to clutter, poor or variable lighting andobject resolutions, distracting competing visual information, etc. Falsealerts or missed event recognitions must also occur at an acceptablelevel.

BRIEF SUMMARY

In one embodiment of the present invention, a method for using modelslearned from anomaly detection to rank detected anomalies includesretrieving anomaly results from an anomaly detection module. The anomalydetection module has local models including image feature valuesextracted from an image field of video image data with respect todifferent predefined spatial and temporal local units, wherein anomalyresults are determined by failures to fit to applied anomaly detectionmodule local models. Thus, the method includes normalizing image featurevalues extracted from the image field local units associated withanomaly results, clustering image feature values extracted from theimage field local units, and learning weights for the anomaly results asa function of the relations of their normalized extracted image featurevalues to the clustered image feature values. The normalized values aremultiplied by the learned weights to generate ranking values to rank theanomalies.

In another embodiment, a system has a processing unit, computer readablememory and a computer readable storage medium device with programinstructions to rank detected anomalies retrieved from an anomalydetection module. The anomaly detection module has local modelsincluding image feature values extracted from an image field of videoimage data with respect to different predefined spatial and temporallocal units, wherein anomaly results are determined by failures to fitto applied anomaly detection module local models. Thus, the systemnormalizes image feature values extracted from the image field localunits associated with anomaly results, clusters image feature valuesextracted from the image field local units, and learns weights for theanomaly results as a function of the relations of their normalizedextracted image feature values to the clustered image feature values.The normalized values are multiplied by the learned weights to generateranking values to rank the anomalies.

In another embodiment, an article of manufacture has a computer readablestorage medium device with computer readable program code embodiedtherewith, the computer readable program code comprising instructionsthat, when executed by a computer processor, cause the computerprocessor to rank detected anomalies retrieved from an anomaly detectionmodule. The anomaly detection module has local models including imagefeature values extracted from an image field of video image data withrespect to different predefined spatial and temporal local units,wherein anomaly results are determined by failures to fit to appliedanomaly detection module local models. Thus, the computer processornormalizes image feature values extracted from the image field localunits associated with anomaly results, clusters image feature valuesextracted from the image field local units, and learns weights for theanomaly results as a function of the relations of their normalizedextracted image feature values to the clustered image feature values.The normalized values are multiplied by the learned weights to generateranking values to rank the anomalies.

In another embodiment, a method for providing a service for using modelslearned from anomaly detection to rank detected anomalies includesproviding one or more components or articles. Thus, a results retrieverretrieves anomaly results from an anomaly detection module, which haslocal models including image feature values extracted from an imagefield of video image data with respect to different predefined spatialand temporal local units, wherein anomaly results are determined byfailures to fit to applied anomaly detection module local models. Apriority learning component normalizes image feature values extractedfrom the image field local units associated with anomaly results,clusters image feature values extracted from the image field localunits, and learns weights for the anomaly results as a function of therelations of their normalized extracted image feature values to theclustered image feature values. A ranker multiplies normalized values bythe learned weights to generate ranking values to rank the anomalies.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

These and other features of this invention will be more readilyunderstood from the following detailed description of the variousaspects of the invention taken in conjunction with the accompanyingdrawings in which:

FIG. 1 is a diagrammatic flow chart illustration of an embodiment of anarticle or system that uses models learned from an anomaly detector torank detected anomalies according to the present invention.

FIG. 2 is a diagrammatic flow chart illustration of an embodiment of amethod or system for priority learning and ranking of detected anomaliesaccording to the present invention.

FIG. 3 is a diagrammatic flow chart illustration of an anomaly detectionmodule or process according to embodiments of the present invention.

FIG. 4 is a graphic illustration of a partitioned video data image fieldaccording to embodiments of the present invention.

FIG. 5 is a block diagram illustration of a computerized implementationof an embodiment of the present invention.

The drawings are not necessarily to scale. The drawings are merelyschematic representations, not intended to portray specific parametersof the invention. The drawings are intended to depict only typicalembodiments of the invention and, therefore, should not be considered aslimiting the scope of the invention. In the drawings, like numberingrepresents like elements.

DETAILED DESCRIPTION

As will be appreciated by one skilled in the art, aspects of the presentinvention may be embodied as a system, method or computer programproduct. Accordingly, aspects of the present invention may take the formof an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, aspects of the present invention may take the form of acomputer program product embodied in one or more computer readablemedium(s) having computer readable program code embodied thereon.

Any combination of one or more computer readable medium(s) may beutilized. The computer readable medium may be a computer readable signalmedium or a computer readable storage medium. A computer readablestorage medium may be, for example, but not limited to, an electronic,magnetic, optical, electromagnetic, infrared, or semiconductor system,apparatus, or device, or any suitable combination of the foregoing. Morespecific examples (a non-exhaustive list) of the computer readablestorage medium would include the following: an electrical connectionhaving one or more wires, a portable computer diskette, a hard disk, arandom access memory (RAM), a read-only memory (ROM), an erasableprogrammable read-only memory (EPROM or Flash memory), an optical fiber,a portable compact disc read-only memory (CD-ROM), an optical storagedevice, a magnetic storage device, or any suitable combination of theforegoing. In the context of this document, a computer readable storagemedium may be any tangible medium that can contain or store a programfor use by or in connection with an instruction execution system,apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, in abaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium may be transmittedusing any appropriate medium, including, but not limited to, wireless,wireline, optical fiber cable, RF, etc., or any suitable combination ofthe foregoing.

Computer program code for carrying out operations for aspects of thepresent invention may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++ or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Aspects of the present invention are described below with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

FIG. 1 is a diagrammatic illustration of one embodiment of an anomalyretriever or system 10 (for example, a programmable device, computersystem, etc.) according to the present invention that uses modelslearned from anomaly detection to rank detected anomalies. In responseto an input query 18 for one or more anomalies, a Results Retriever 20retrieves a plurality of anomaly results provided through use of anassociated Anomaly Detection system or component 12, which in thepresent example has both a Local Model 16 and a Global Model 14, thoughother embodiments may use more or less, or only one, of the Local Model16 and the Global Model 14, or use different anomaly detection models.

The anomaly results retrieved from anomaly detection at 12 are ranked(or prioritized) by a Ranker 22 as a function of weightings learned byan online Priority Learning Component 24, which learns the weightingsthrough clustering features extracted by the Anomaly Detector 12 andfurther determines importance and relevance parameters with respect toassociated local areas of the field of image of the video data in viewof learned models of the Anomaly Detector 12. The retrieved results arepresented as a function of their ranking by the Ranker 22 to a usersubmitting the query 18 in a Retrieval Front End 26, for example in abrowser or other user interface.

FIG. 2 is a diagrammatic flow chart illustration of one embodiment of amethod or system for priority learning and ranking of detected anomaliesaccording to the present invention. Thus, at 50 a plurality of anomalyresults are retrieved from an anomaly detection module comprising aplurality of local models in response to an input query for an anomaly.The anomaly detection module local models comprise image feature valuesextracted from an image field of video image data with respect to eachof a plurality of different predefined spatial and temporal local units,and each of the plurality of anomaly results are determined byrespective failures to fit to applied ones of the anomaly detectionmodule local models.

At 52 image features extracted from the image field local units that areassociated with each of the plurality of anomaly results are normalized.At 54 image feature values extracted from the each image field localunits that are associated with the each of the plurality of anomalyresults are clustered. At 56 weights for each of the anomaly results arelearned as a function of a relation of their normalized extracted imagefeature values to the clustered image feature values. At 58 the anomalynormalized extracted feature values are multiplied by their respectivelearned weights to generate respective ranking values, and the anomaliesand thus ranked by their respective generated ranking values at 60.

FIG. 3 is a diagrammatic illustration of one embodiment of a localmodel/global model detector structure 212 appropriate for performing theanomaly detection at 12 of FIG. 1. In the local anomaly detection model14 an image field of a video data input 102 is partitioned into aplurality of different predefined spatial and temporal local areas orunits, wherein at 104 a local detector extracts and clusters imagefeatures with respect to each unit. Embodiments of the present inventiondo not require prior knowledge of either normal or abnormal patterns,but instead they may automatically learn normal patterns by learningdominant behaviors from the extracted and clustered features.

Various extracted video features appropriate for use in embodiments ofthe present invention include, but are not limited to, color, motion,texture, edges, etc., and may be extracted from each local unit andfurther refined by a dimension reduction technique. Dominantdistributions of the extracted features are found and used to define“normal” patterns, wherein rare patterns define “anomaly” or “abnormal”patterns. Learned models are constructed by building either parametricmodels (for example, Gaussian) or non-parametric models (for example,kernel density estimation) for the learned feature distributions.Extracted local unit local image features may be fit to the models 108to “learn” revisions, for example to align to clustered similarities ofnew extracted features.

At 106 the image features extracted by a local detector for each localunit are compared to learned local motion pattern models 108 for eachlocal unit to generate local anomaly detection confidence decisionvalues at 110 for each of said local units, more particularly whetherthe features extracted relevant to object motion within input video dataindicate that the object motion within each particular local unit iseither normal or expected, or instead anomalous (abnormal orunexpected), in view of the learned model patterns for the local units.Local global anomaly detection confidence decision values 110 may bebinary normal or anomaly values (i.e. “yes” or “no”, or “one” or“zero”), or they be graduated values or other non-binary values. Thus,anomaly detection decisions are made at 110 based on individual gridfitting confidences, and an anomaly decision may be made for each localunit. Each local unit may be assigned a label, with the distribution ofthe labels giving information content of the local unit. Internalappearance patterns (for example, entropy) may be used as a measure tofind the significance metric of each such local unit, which may beembedded in the anomaly detection models 14/16 to improve them, and alsoused to prioritize the anomalies by revising weightings used in rankingat 22.

In the global anomaly detection module 14, at 112 the presence of anobject in an image field of the video data input 102 is detected and itsmovement tracked through the image field over time through a trajectoryof motion, for example through background modeling and subtractionprocesses, though other techniques may be practiced. Illustrative butnot exhaustive tracked movement examples include a person object movingrelative to (for example, travelling up) a staircase object and turningdown a hallway object, and observing a changing separation value betweentwo structural elements in an assembly over time that may be indicativeof a structural change of the assembly or elements. At 114 a globalfeature extractor extracts image features from the video data relativeto the trajectory of the object tracked through the image field withrespect to all or a portion of the image field. At 118 a global anomalydetector compares the extracted trajectory features to a learned motiontrajectory model 116 to generate a global anomaly detection confidencedecision values 120 for the object trajectory: for example, whether thetrajectory fits to a normal learned trajectory, or not. The decisionvalue at 120 provides an objective measure of likelihood that the objecttrajectory is either normal or instead anomalous.

At 124 the system or process decides whether or not an anomaly hasoccurred as a function of the individual local unit local anomalydetection confidence decision values 110 and the global anomalydetection confidence values 120. The decision at 124 may be based on theindividual local and global anomaly detection confidence values 110/120,or through a combination or fusion of the respective values 110/120, forexample fusing the values 110 for each of the grids that an objectpasses through in a trajectory with the global value 120 for thetrajectory to provide a fused value.

FIG. 4 illustrates an example of an image field 402 of input video data102 which comprises images of objects that describe a trajectory ofmotion 406 over time (for example, a person travelling along a concoursewithin the image field 402). The image field 402 is divided into amatrix of predefined spatial and temporal local units or grids 404. Theobject trajectory 406 travels through some of the grids 404 a, but doesnot enter into other grids 404 b. In one embodiment, the local anomalydetection confidence values 110 of the grids 404 a that include thetracked trajectory 406 are combined or fused with the global anomalydetection confidence value 120 of the tracked trajectory 406 to decideat 124 if the object movement is normal or an anomaly. A total number ofthe image field grids 404 a that include the object trajectory may beless than a totality of all of the partition grids 404 (inclusive ofsaid grids 404 a and the other grids 404 b), leading to efficienciesover other systems that may extract features for every one of the grids404.

At 126 the anomaly detection module 212 updates or refines, or buildsnew models, for the local and/or global learned models 108 and/or 116 asa function of the anomaly decisions at 124 through analyzing activitypatterns from the video image data input 102, and/or through feedback orother data from the priority learning at 24 of the anomaly retriever 10.Analysis at 126 may be carried on in different scales, both in the localand global levels of the video, and in both spatial and temporaldomains.

Rankings are accomplished at 22 using a variety of context measures, andas a function of clustering of extracted feature values (in someembodiments, as a function of comparing the clusters with the learnedlocal model 108 local units), and of weightings learned by the prioritylearning at 24. Various ranking or weighting mechanisms may be used tocompute final ranking scores through considering and combining thevarious features. In one embodiment, a linear weighted sum is used,wherein all the extracted features (F_(i)) are normalized to havefeature values between zero and one, and weights (W_(i)) provided by thepriority learning at 24 are chosen pursuant to equation [1]:

$\begin{matrix}{{\sum\limits_{i}W_{i}} = 1.} & \lbrack 1\rbrack\end{matrix}$

For each detected anomaly a relevance measure ranking is computed usingformulation [2]:

$\begin{matrix}{{\sum\limits_{i}{W_{i}*F_{i}}};} & \lbrack 2\rbrack\end{matrix}$

wherein the retrieved results are displayed at 26 (FIG. 1) in the orderof relevance measure.

Embodiments of the present invention provide for online prioritylearning at 24 to learn the weights (W_(i)) from features captured fromthe underlying activity that may include subjective information. Someembodiments use a linear weighted function based on feedback fromanomaly detection processes or components or from user inputs to “learn”updates to the weights (W_(i)), thus to make them better over time.Extracted clusters and the cluster scores carry salient informationabout the underlying activity and how aberrant it is to otheractivities. In one aspect, embodiments that divide anomaly detectioninto separate local and global processes allow for the categorization ofanomalies in a better manner over conventional anomaly detectionmethods, through enabling the imposition of spatial, temporal andspatio-temporal constraints to learn the weights (W_(i)) used to rankthe results.

For example, the weights (W_(i)) may be revised through prioritylearning by determining a spatial location of a retrieved anomaly withinthe field of view of the input video data as correlated to features ofinterest of the real-world scene represented within the field of viewand assign a ranking weighting accordingly. Thus, a firstcluster-outlier anomaly may receive a higher weight (W_(i)) to achieve ahigher ranking (or ranking metric value) if its spatial location iswithin a portion of the field of view of the input video that iscorrelated with a cordoned off area of the real-world scene representedwithin the field of view (for example, the rails of a train within apassenger station), as compared with the weight (W_(i)) assigned toanother second anomaly that is also a cluster outlier (perhaps having anequivalent distance to a center of a same cluster of object motionevents) but is spatially outside the portion (and thus not in the railarea). Object activity within this portion area is thus predetermined tobe more concerning than anomaly activity outside of it.

The weights may also be a function of distance to centers of clusteredevents, or of clusters of other anomalies. Thus, anomalies occurringwithin an image field outside of a cluster and farther from the clustercenter area of the image may be ranked proportionally higher than thosethat are closer when the context of the object activity indicates thatthe greater spatial distance occurrences are more concerning; in oneexample, the greater distance may suggest that a detected person objecthas removed himself from the sight of others in order to engage in anillicit activity. Accordingly, anomalies which are more distant (C_(i))from centers of clusters of the extracted features may also be givenhigher weights (W_(i)) compared to the ones closer according toformulation [3]:

W _(i) =|C _(i) −F _(i)|².  [3]

Or, in the converse, anomalies occurring within an image field outsideof a cluster and closer to the cluster center area of the image may beweighted proportionally higher (hence, to be ranked higher) than thosethat are both outside of the cluster and farther from the clustercenter.

The distribution of a plurality of feature clusters may also be used toinfer how frequently anomalies occur and thus to assign respectiveweights (W_(i)). Temporal data such as duration or time of day may alsobe used to rank or weight anomalies. For example, anomalies at night maybe more concerning than daytime activities (which are more likely to benormal or non-concealed activities). Activities of longer duration maybe ranked or weighted higher: a longer running time in a station areamay indicate a fleeing activity, rather a short run to catch a train.

Frequency of anomaly occurrence may also be used to rank or weight.Thus, a rarely occurring anomaly (for example, movement in a subset areaof a restricted area that is rarely occupied), or a frequently occurringanomaly (for example, one that is more strongly correlated with anactivity of concern than another anomaly) may merit enhanced attention.

Weights may also reflect camera priority. Thus, each of a plurality ofdifferent cameras may have different weights representative ofpriorities assigned to associated data. In one embodiment, with regardto reporting an anomaly of a running person, a camera facing subwaytracks is ranked higher than a camera facing a turnstile area. A type ofthe anomaly may also merit different rankings/weightings based on itscontext: placing a large bag in a refuse receptacle may not beconcerning with respect to a building dumpster, but in a public trashcan in an assembly area, where occupants are not expected to have largebags of rubbish to dispose of, would be ranked/weighted higher.

In prior art large video data implementations with large pluralities ofcameras, anomaly detection is generally performed with respect toindividual camera data without communication between cameras. Incontrast, embodiments of the present invention enable a user to select aregion or subset set of cameras through the front-end retrievalinterface 26 for anomaly retrieval and ranking to across the selectedcameras, and wherein the priority learning component or process mayincrease weightings (and thus rankings) of anomalies across the multipleselected cameras. For example, where one anomaly near a secure zone isconsidered more important than tens of anomalies in another area,priority learning may automatically adjust the rankings of suchanomalies upward. In one embodiment, the weights (W_(i)) assigned toanomalies from a given camera are determined as a function of a priorimportance value (PI_(i)) given to the camera based on the type ofanomalies that can be expected to occur, multiplied by an updatedimportance value (UI_(i)) for the camera based on the type of anomaliesand number of anomalies that are observed to occur in the camera byanomaly detection, the priority learning thus continually updating theassigned weight (W_(i)) in an ongoing learning process in proportion toits initial importance.

A variety of methods and processes may be used for feature detection andextraction according to embodiments of the present invention. In oneembodiment, feature vectors comprising ten dimensions are utilizedwithin the local and global models 14/16, eight for directionalcomponents and two for velocity in horizontal and vertical directions,wherein spatiotemporal feature vectors are derived therefrom (forexample, by concatenating the directional vectors over a number of videoimage frames); wherein Matlab™ or Principal Component Analysis (PCA) isused to reduce dimensionality; and agglomerative clustering of thefeature vectors is used (which in one aspect helps in providing ahierarchy). MATLAB is a trademark of The MathWorks, Inc., in the UnitedStates or other countries.

Thus, embodiments of the present invention learn the significance ofevents that are spatial and temporal in nature and use the learnedsignificance measure(s) to rank or prioritize retrieved anomalies toprovide more efficient anomaly detection. Information theory techniques(for example, Entropy, Kolomogorov property, etc.) may be used toextract significance measure metrics (such as the relevance score(R_(i)) discussed above) for each local unit, gird or other field ofview partition. In one aspect, such significance measurement metricsessentially magnify the anomalies retrieved with respect to local unitswhere the probability of an anomaly occurring is high, and subdue theanomalies retrieved with respect to the other local units throughlowered rankings, and wherein the significance measure(s) may be updatedonline instead of during other, offline training.

Standard prior art metrics for analyzing and prioritizing anomalies suchas mean, median, frequency will not provide comparable rankings as theyare not responsive to, nor do they capture, the underlying activity. Incontrast, embodiments of the present invention that use local and globallearned models enable better ranking of the anomalies, as the underlyinganomaly clusters and their distribution contain valuable informationthat is incorporated into prioritizing results for better retrieval. Byusing global trajectories and local motion patterns, certain type ofanomalies may be prioritized over others according to the needs of theend user.

Referring now to FIG. 5, an exemplary computerized implementation of anembodiment of the present invention includes computer or otherprogrammable device 522 in communication with other devices 506 (forexample, a video camera or video server, or a memory device comprising adatabase of images, etc.) that uses models learned from anomalydetection to rank detected anomalies as described above with respect toFIGS. 1 through 4, for example in response to instructions 542 withincomputer readable code residing in a computer memory 516, or in thestorage system 532, another device 506 or other computer readablestorage medium that is accessed through a computer networkinfrastructure 526. Thus, the instructions, when implemented in aprocessing unit (CPU) 538 may provide anomaly detection throughcombining outputs from local and global modules as described above withrespect to FIGS. 1-4.

The computer 522 comprises various components, some of which areillustrated within the computer 522. More particularly, as shown, thecomputer 522 includes a processing unit (CPU) 538 in communication withone or more external I/O devices/resources 524, storage systems 532 orother devices 520. Moreover, the processing unit 538 may comprise asingle processing unit, or be distributed across one or more processingunits in one or more locations, e.g., on a client and server. Similarly,the memory 516 and/or the storage system 532 can comprise anycombination of various types of data storage and/or transmission mediathat reside at one or more physical locations. Further, I/O interfaces524 can comprise any system for exchanging information with one or moreof an external server and/or client (not shown). Still further, it isunderstood that one or more additional components (e.g., systemsoftware, math co-processing unit, etc.), not shown, can be included inthe computer 522.

Embodiments of the present invention may also perform process steps ofthe invention on a subscription, advertising, and/or fee basis. That is,a service provider could offer to use models learned from anomalydetection to rank detected anomalies as described above with respect toFIGS. 1-5. Thus, the service provider can create, maintain, and support,etc., a computer infrastructure, such as the network computer system522, or network environment 526 (or parts thereof) that perform theprocess steps of the invention for one or more customers. In return, theservice provider can receive payment from the customer(s) under asubscription and/or fee agreement and/or the service provider canreceive payment from the sale of advertising content to one or morethird parties. Services may comprise one or more of: (1) installingprogram code on a computing device, such as the computers/devices 522,from a computer-readable medium device 516, 520 or 506; (2) adding oneor more computing devices to a computer infrastructure; and (3)incorporating and/or modifying one or more existing systems of thecomputer infrastructure to enable the computer infrastructure to performthe process steps of the invention.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof. Certain examples and elementsdescribed in the present specification, including in the claims and asillustrated in the Figures, may be distinguished or otherwise identifiedfrom others by unique adjectives (e.g. a “first” element distinguishedfrom another “second” or “third” of a plurality of elements, a “primary”distinguished from a “secondary” one or “another” item, etc.) Suchidentifying adjectives are generally used to reduce confusion oruncertainty, and are not to be construed to limit the claims to anyspecific illustrated element or embodiment, or to imply any precedence,ordering or ranking of any claim elements, limitations or process steps.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed.

Many modifications and variations will be apparent to those of ordinaryskill in the art without departing from the scope and spirit of theinvention. The embodiment was chosen and described in order to bestexplain the principles of the invention and the practical application,and to enable others of ordinary skill in the art to understand theinvention for various embodiments with various modifications as aresuited to the particular use contemplated.

1. A method for using models learned from anomaly detection to rankdetected anomalies, the method comprising: retrieving a plurality ofanomaly results from an anomaly detection module comprising a pluralityof local models in response to an input query for an anomaly, whereinthe anomaly detection module local models comprise image feature valuesextracted from an image field of video image data with respect to eachof a plurality of different predefined spatial and temporal local units,and each of the plurality of anomaly results are determined byrespective failures to fit to applied ones of the anomaly detectionmodule local models; normalizing each of a plurality of values of imagefeatures extracted from the image field local units and that areassociated with each of the plurality of anomaly results; clusteringimage feature values extracted from the each image field local unitsthat are associated with the each of the plurality of anomaly results;learning each of a plurality of weights for each of the anomaly resultsas a function of a relation of their normalized values of the extractedimage features to the clustered image feature values extracted from theeach respective associated image field local units; multiplying thenormalized values of extracted features of each of the plurality ofanomalies by their respective learned weights to generate respectiveranking values; and ranking the plurality of anomalies by theirgenerated respective ranking values.
 2. The method of claim 1, whereinthe step of learning each of the plurality of weights for each of theanomaly results as a function of the relation of their normalized valuesof the extracted image features to the clustered image feature valuesextracted from the each respective associated image field local unitsfurther comprises: determining spatial locations of each of theretrieved anomalies within the field of view of the input video data ascorrelated to features of interest of a real-world scene representedwithin the field of view; and assigning a first weighting to a first ofthe retrieved anomalies that is higher than a second weighting assignedto a second of the retrieved anomalies in response to the determinedspatial location of the first anomaly being within a portion of thefield of view of the input video that is correlated with a cordoned offarea of the real-world scene and the determined spatial location of thesecond anomaly being outside the portion; and wherein the normalizedextracted features of each of the first and second anomalies areoutliers from and have the same distance to a center of a same learnedmodel cluster of extracted features of a one of the anomaly detectionmodule local models.
 3. The method of claim 2, further comprising:updating the weighting of one of the first and second anomalies byincreasing or decreasing the updated weighting relative to the weightingof the other of the first and second anomalies as a function of adifference in relative distances of respective normalized extractedfeatures from a center of a same associated learned model cluster of theextracted features.
 4. The method of claim 3, further comprising:updating the weighting of one of the first and second anomalies byincreasing or decreasing the updated weighting relative to the weightingof the other of the first and second anomalies as a function of adifference in temporal data of the respective first and secondanomalies.
 5. The method of claim 4, wherein the difference in thetemporal data is a longer duration time of an associated tracked objectmovement.
 6. The method of claim 4, wherein the difference in thetemporal data indicates a difference in a time of day of occurrences ofthe respective first and second anomalies.
 7. The method of claim 4,further comprising: assigning a prior importance value to a cameraassociated with one of the first and second anomalies based on a type ofanomaly expected to occur in video data from the camera; generating anupdated importance value for the camera based on at least one of a typeof observed anomaly or a number of observed anomalies that occur in thevideo data from the camera as determined by anomaly detection; andupdating the weighting of the one of the first and second anomaliesassociated with the camera by increasing or decreasing the weightingrelative to the weighting of the other of the first and second anomaliesas a function of a product of the prior importance value multiplied bythe updated importance value.
 8. A system, comprising: a processingunit, computer readable memory and a computer readable storage medium;first program instructions to retrieve a plurality of anomaly resultsfrom an anomaly detection module comprising a plurality of local modelsin response to an input query for an anomaly, wherein the anomalydetection module local models comprise image feature values extractedfrom an image field of video image data with respect to each of aplurality of different predefined spatial and temporal local units, andeach of the plurality of anomaly results are determined by respectivefailures to fit to applied ones of the anomaly detection module localmodels; second program instructions to normalize each of a plurality ofvalues of image features extracted from the image field local units andthat are associated with each of the plurality of anomaly results; thirdprogram instructions to cluster image feature values extracted from theeach image field local units that are associated with the each of theplurality of anomaly results, and to learn each of a plurality ofweights for each of the anomaly results as a function of a relation oftheir normalized values of the extracted image features to the clusteredimage feature values extracted from the each respective associated imagefield local units; and fourth program instructions to multiply thenormalized values of extracted features of each of the plurality ofanomalies by their respective learned weights to generate respectiveranking values and rank the plurality of anomalies by their generatedrespective ranking values; and wherein the first, second, third andfourth program instructions are stored on the computer readable storagemedium for execution by the processing unit via the computer readablememory.
 9. The system of claim 8, wherein the third program instructionsare further to learn each of the plurality of weights for each of theanomaly results as a function of the relation of their normalized valuesof the extracted image features to the clustered image feature valuesextracted from the each respective associated image field local unitsby: determining spatial locations of each of the retrieved anomalieswithin the field of view of the input video data as correlated tofeatures of interest of a real-world scene represented within the fieldof view; and assigning a first weighting to a first of the retrievedanomalies that is higher than a second weighting assigned to a second ofthe retrieved anomalies in response to the determined spatial locationof the first anomaly being within a portion of the field of view of theinput video that is correlated with a cordoned off area of thereal-world scene and the determined spatial location of the secondanomaly being outside the portion; and wherein the normalized extractedfeatures of each of the first and second anomalies are outliers from andhave the same distance to a center of a same learned model cluster ofextracted features of a one of the anomaly detection module localmodels.
 10. The system of claim 9, wherein the third programinstructions are further to update the weighting of one of the first andsecond anomalies by increasing or decreasing the updated weightingrelative to the weighting of the other of the first and second anomaliesas a function of a difference in relative distances of respectivenormalized extracted features from a center of a same associated learnedmodel cluster of the extracted features.
 11. The system of claim 10,wherein the third program instructions are further to update theweighting of one of the first and second anomalies by increasing ordecreasing the updated weighting relative to the weighting of the otherof the first and second anomalies as a function of a difference intemporal data of the respective first and second anomalies.
 12. Thesystem of claim 11, wherein the third program instructions are furtherto: assign a prior importance value to a camera associated with one ofthe first and second anomalies based on a type of anomaly expected tooccur in video data from the camera; generate an updated importancevalue for the camera based on at least one of a type of observed anomalyor a number of observed anomalies that occur in the video data from thecamera as determined by anomaly detection; and update the weighting ofthe one of the first and second anomalies associated with the camera byincreasing or decreasing the weighting relative to the weighting of theother of the first and second anomalies as a function of a product ofthe prior importance value multiplied by the updated importance value.13. An article of manufacture, comprising: a computer readable storagemedium having computer readable program code embodied therewith, thecomputer readable program code comprising instructions that, whenexecuted by a computer processor, cause the computer processor to:retrieve a plurality of anomaly results from an anomaly detection modulecomprising a plurality of local models in response to an input query foran anomaly, wherein the anomaly detection module local models compriseimage feature values extracted from an image field of video image datawith respect to each of a plurality of different predefined spatial andtemporal local units, and each of the plurality of anomaly results aredetermined by respective failures to fit to applied ones of the anomalydetection module local models; normalize each of a plurality of valuesof image features extracted from the image field local units and thatare associated with each of the plurality of anomaly results; clusterimage feature values extracted from the each image field local unitsthat are associated with the each of the plurality of anomaly results;learn each of a plurality of weights for each of the anomaly results asa function of a relation of their normalized values of the extractedimage features to the clustered image feature values extracted from theeach respective associated image field local units; multiply thenormalized values of extracted features of each of the plurality ofanomalies by their respective learned weights to generate respectiveranking values; and rank the plurality of anomalies by their generatedrespective ranking values.
 14. The article of manufacture of claim 13,wherein the computer readable program code instructions, when executedby the computer processor, further cause the computer processor to learneach of the plurality of weights for each of the anomaly results as afunction of the relation of their normalized values of the extractedimage features to the clustered image feature values extracted from theeach respective associated image field local units by: determiningspatial locations of each of the retrieved anomalies within the field ofview of the input video data as correlated to features of interest of areal-world scene represented within the field of view; and assigning afirst weighting to a first of the retrieved anomalies that is higherthan a second weighting assigned to a second of the retrieved anomaliesin response to the determined spatial location of the first anomalybeing within a portion of the field of view of the input video that iscorrelated with a cordoned off area of the real-world scene and thedetermined spatial location of the second anomaly being outside theportion; and wherein the normalized extracted features of each of thefirst and second anomalies are outliers from and have the same distanceto a center of a same learned model cluster of extracted features of aone of the anomaly detection module local models.
 15. The article ofmanufacture of claim 14, wherein the computer readable program codeinstructions, when executed by the computer processor, further cause thecomputer processor to update the weighting of one of the first andsecond anomalies by increasing or decreasing the updated weightingrelative to the weighting of the other of the first and second anomaliesas a function of a difference in relative distances of respectivenormalized extracted features from a center of a same associated learnedmodel cluster of the extracted features.
 16. The article of manufactureof claim 15, wherein the computer readable program code instructions,when executed by the computer processor, further cause the computerprocessor to update the weighting of one of the first and secondanomalies by increasing or decreasing the updated weighting relative tothe weighting of the other of the first and second anomalies as afunction of a difference in temporal data of the respective first andsecond anomalies.
 17. The article of manufacture of claim 16, whereinthe computer readable program code instructions, when executed by thecomputer processor, further cause the computer processor to: assign aprior importance value to a camera associated with one of the first andsecond anomalies based on a type of anomaly expected to occur in videodata from the camera; generate an updated importance value for thecamera based on at least one of a type of observed anomaly or a numberof observed anomalies that occur in the video data from the camera asdetermined by anomaly detection; and update the weighting of the one ofthe first and second anomalies associated with the camera by increasingor decreasing the weighting relative to the weighting of the other ofthe first and second anomalies as a function of a product of the priorimportance value multiplied by the updated importance value.
 18. Amethod for providing a service for using models learned from anomalydetection to rank detected anomalies, the method comprising: providing aresults retriever that retrieves a plurality of anomaly results from ananomaly detection module comprising a plurality of local models inresponse to an input query for an anomaly, wherein the anomaly detectionmodule local models comprise image feature values extracted from animage field of video image data with respect to each of a plurality ofdifferent predefined spatial and temporal local units, and each of theplurality of anomaly results are determined by respective failures tofit to applied ones of the anomaly detection module local models;providing a priority learning component that normalizes each of aplurality of values of image features extracted from the image fieldlocal units and that are associated with each of the plurality ofanomaly results, clusters image feature values extracted from the eachimage field local units that are associated with the each of theplurality of anomaly results, learns each of a plurality of weights foreach of the anomaly results as a function of a relation of theirnormalized values of the extracted image features to the clustered imagefeature values extracted from the respective associated each image fieldlocal units; and provides a ranker that multiplies the normalized valuesof extracted features of each of the plurality of anomalies by theirrespective learned weights to generate respective ranking values andranks the plurality of anomalies by their generated respective rankingvalues.
 19. The method of claim 13, wherein the priority learningcomponent learns each of the plurality of weights for each of theanomaly results as a function of the relation of their normalized valuesof the extracted image features to the clustered image feature valuesextracted from the respective associated each image field local unitsby: determining spatial locations of each of the retrieved anomalieswithin the field of view of the input video data as correlated tofeatures of interest of a real-world scene represented within the fieldof view; and assigning a first weighting to a first of the retrievedanomalies that is higher than a second weighting assigned to a second ofthe retrieved anomalies in response to the determined spatial locationof the first anomaly being within a portion of the field of view of theinput video that is correlated with a cordoned off area of thereal-world scene and the determined spatial location of the secondanomaly being outside the portion; and wherein the normalized extractedfeatures of each of the first and second anomalies are outliers from andhave the same distance to a center of a same learned model cluster ofextracted features of a one of the anomaly detection module localmodels.
 20. The method of claim 19, wherein the priority learningcomponent updates the weighting of one of the first and second anomaliesby increasing or decreasing the updated weighting relative to theweighting of the other of the first and second anomalies as a functionof a difference in relative distances of respective normalized extractedfeatures from a center of a same associated learned model cluster of theextracted features.
 21. The method of claim 20, wherein the prioritylearning component updates the weighting of one of the first and secondanomalies by increasing or decreasing the updated weighting relative tothe weighting of the other of the first and second anomalies as afunction of a difference in temporal data of the respective first andsecond anomalies.
 22. The method of claim 21, wherein the prioritylearning component further: assigns a prior importance value to a cameraassociated with one of the first and second anomalies based on a type ofanomaly expected to occur in video data from the camera; generates anupdated importance value for the camera based on at least one of a typeof observed anomaly or a number of observed anomalies that occur in thevideo data from the camera as determined by anomaly detection; andupdates the weighting of the one of the first and second anomaliesassociated with the camera by increasing or decreasing the weightingrelative to the weighting of the other of the first and second anomaliesas a function of a product of the prior importance value multiplied bythe updated importance value.